摘要 :
As the test effort takes a significant part of the software development lifecycle, efficient test strategies are a precondition for reduction of development costs and time. In this respect two main issues exist: firstly, the tunin...
展开
As the test effort takes a significant part of the software development lifecycle, efficient test strategies are a precondition for reduction of development costs and time. In this respect two main issues exist: firstly, the tuning of the test track from test case identification to evaluation, secondly, the reduction of number of test cases to be processed and evaluated. Both aspects were considered in the work presented in this paper. For reduction of the effort related to the test track two test automation tools have been applied: DCRTT for C and SmartG for Java. While DCRTT is ready for industrial use at high degree of automation of all test steps, SmartG is a prototype exploiting the identifcation of path sets by random testing. DCRTT only requires provision of the source files and then delivers test drivers, a filtered and reduced set of test cases, related test results and detailed information on data ranges and observed exceptions. The manual effort is reduced to test evaluation. DCRTT identifies a significantly reduced set of test cases left for manual evaluation. The selection criteria are based on block coverage, decision coverage and occurrence of exceptions when generating inputs from the valid and invalid data range. One or more test cases may be collected for each basket of such a criterion and element of a function's structure. It is of high importance how well such automated strategies do work. Therefore a number of investigations have been performed, evaluating the achieved coverage and number of reported exceptions. Three test modes have been considered: random and lattice-based test generation for module testing and operational testing imposing the complete main program to representative operational conditions.
收起
摘要 :
Standards are used to describe and ensure the quality of
products, services and processes throughout almost all
branches of industry, including the field of software
engineering. Contractors and suppliers are obligated by
thei...
展开
Standards are used to describe and ensure the quality of
products, services and processes throughout almost all
branches of industry, including the field of software
engineering. Contractors and suppliers are obligated by
their customers and certification authorities to follow a
certain set of standards during development. For
example, a customer can easier actively participate in
and control the contractor’s process when enforcing a
standard process..
However, as with any requirement, a standard may also
impede the contractor or supplier in assuring actual
quality of the product in the sense of fitness for the
purpose intended by the customer.
This is the case when a standard defines specific quality
assurance activities requiring a considerable amount of
effort while other more efficient but equivalent or even
superior approaches are blocked. Then improvement of
the ratio between cost and quality exceeding miniscule
advances is heavily impeded.
While in some parts being too specific in defining the
mechanisms of the enforced process, standards are
sometimes too weak in defining the principles or goals
on control of product quality.
Therefore this paper addresses the following issues: (1)
Which conclusions can be drawn on the quality and
efficiency of a standard? (2) If and how is it possible to
improve or evolve a standard? (3) How well does a
standard guide a user towards high quality of the end
product?
One conclusion is that the analyzed standards do
interfere with technological innovation, though the
standards leave a lot of freedom for concretization and
are understood as technology-independent.
Another conclusion is that standards are not only a
matter of quality but also a matter of competitiveness of
the industry depending on resulting costs and time-tomarket.
When the costs induced by a standard are not
adequate to the achievable quality, industry encounters a
significant disadvantage.
收起
摘要 :
Standards are used to describe and ensure the quality of products, services and processes throughout almost all branches of industry, including the field of software engineering. Contractors and suppliers are obligated by their cu...
展开
Standards are used to describe and ensure the quality of products, services and processes throughout almost all branches of industry, including the field of software engineering. Contractors and suppliers are obligated by their customers and certification authorities to follow a certain set of standards during development. For example, a customer can easier actively participate in and control the contractor’s process when enforcing a standard process.. However, as with any requirement, a standard may also impede the contractor or supplier in assuring actual quality of the product in the sense of fitness for the purpose intended by the customer. This is the case when a standard defines specific quality assurance activities requiring a considerable amount of effort while other more efficient but equivalent or even superior approaches are blocked. Then improvement of the ratio between cost and quality exceeding miniscule advances is heavily impeded. While in some parts being too specific in defining the mechanisms of the enforced process, standards are sometimes too weak in defining the principles or goals on control of product quality. Therefore this paper addresses the following issues: (1) Which conclusions can be drawn on the quality and efficiency of a standard? (2) If and how is it possible to improve or evolve a standard? (3) How well does a standard guide a user towards high quality of the end product? One conclusion is that the analyzed standards do interfere with technological innovation, though the standards leave a lot of freedom for concretization and are understood as technology-independent. Another conclusion is that standards are not only a matter of quality but also a matter of competitiveness of the industry depending on resulting costs and time-tomarket. When the costs induced by a standard are not adequate to the achievable quality, industry encounters a significant disadvantage.
收起
摘要 :
Standards are used to describe and ensure the quality of products, services and processes throughout almost all branches of industry, including the field of software engineering. Contractors and suppliers are obligated by their cu...
展开
Standards are used to describe and ensure the quality of products, services and processes throughout almost all branches of industry, including the field of software engineering. Contractors and suppliers are obligated by their customers and certification authorities to follow a certain set of standards during development. For example, a customer can easier actively participate in and control the contractor's process when enforcing a standard process.. However, as with any requirement, a standard may also impede the contractor or supplier in assuring actual quality of the product in the sense of fitness for the purpose intended by the customer. This is the case when a standard defines specific quality assurance activities requiring a considerable amount of effort while other more efficient but equivalent or even superior approaches are blocked. Then improvement of the ratio between cost and quality exceeding miniscule advances is heavily impeded. While in some parts being too specific in defining the mechanisms of the enforced process, standards are sometimes too weak in defining the principles or goals on control of product quality. Therefore this paper addresses the following issues: (1) Which conclusions can be drawn on the quality and efficiency of a standard? (2) If and how is it possible to improve or evolve a standard? (3) How well does a standard guide a user towards high quality of the end product? One conclusion is that the analyzed standards do interfere with technological innovation, though the standards leave a lot of freedom for concretization and are understood as technology-independent. Another conclusion is that standards are not only a matter of quality but also a matter of competitiveness of the industry depending on resulting costs and time-to-market. When the costs induced by a standard are not adequate to the achievable quality, industry encounters a significant disadvantage.
收起
摘要 :
Standards are used to describe and ensure the quality of products, services and processes throughout almost all branches of industry, including the field of software engineering. Contractors and suppliers are obligated by their cu...
展开
Standards are used to describe and ensure the quality of products, services and processes throughout almost all branches of industry, including the field of software engineering. Contractors and suppliers are obligated by their customers and certification authorities to follow a certain set of standards during development. For example, a customer can easier actively participate in and control the contractor's process when enforcing a standard process..
However, as with any requirement, a standard may also impede the contractor or supplier in assuring actual quality of the product in the sense of fitness for the purpose intended by the customer.
This is the case when a standard defines specific quality assurance activities requiring a considerable amount of effort while other more efficient but equivalent or even superior approaches are blocked. Then improvement of the ratio between cost and quality exceeding miniscule advances is heavily impeded.
While in some parts being too specific in defining the mechanisms of the enforced process, standards are sometimes too weak in defining the principles or goals on control of product quality.
Therefore this paper addresses the following issues: (1) Which conclusions can be drawn on the quality and efficiency of a standard? (2) If and how is it possible to improve or evolve a standard? (3) How well does a standard guide a user towards high quality of the end product?
One conclusion is that the analyzed standards do interfere with technological innovation, though the standards leave a lot of freedom for concretization and are understood as technology-independent.
Another conclusion is that standards are not only a matter of quality but also a matter of competitiveness of the industry depending on resulting costs and time-to-market. When the costs induced by a standard are not adequate to the achievable quality, industry encounters a significant disadvantage.
收起
摘要 :
"ScaPable" is an acronym derived from "scalable" and "portable". The attribute "scalable"
indicates that specific application software can automatically be built from scratch and
verified without writing any statement in a progr...
展开
"ScaPable" is an acronym derived from "scalable" and "portable". The attribute "scalable"
indicates that specific application software can automatically be built from scratch and
verified without writing any statement in a programming language like C, thereby covering
a large variety of embedded and/or distributed applications. The term "portable"
addresses the capability to automatically port parts of such an application from one
physical node to another one - the processor and operating system type may change -
only requiring the names of the nodes, their processor type and operating system. This
way the infrastructure of an embedded / distributed system can be built just by provision of
literals and figures which define the system interaction, communication, topology and
performance. Moreover, dedicated application software like needed for on-board
command handling, data acquisition and processing, and telemetry handling can be built
from generic templates. The generation time range from less than one second up to about
twenty minutes on a PC/Linux platform (800 MHz). By this extremely short generation time
risks can be identified early because the executable application is immediately available
for validation. A rough estimation shows that one hour of automated system and software
generation is equivalent to about 5 .. 50 man years. Currently, about 50% of a typical
space embedded system can be covered by the available automated approach. However,
the more it is applied, the more can be covered by automation. A system is constructed by
applying a formal transformation to the few information as delivered by the user. This
approach is not limited to the space domain, although the first industrial application was a
space project. Quite different domains can take advantage of such principles of system
construction. This paper explains the approach, compares it with other approaches, and
provides figures on productivity, duration of system generation and reliability.
收起
摘要 :
Performance engineering aims to demonstrate that the software being developed will meet the performance needs. The goal of robustness engineering is to prove that the system will work correctly in the presence or after occurence o...
展开
Performance engineering aims to demonstrate that the software being developed will meet the performance needs. The goal of robustness engineering is to prove that the system will work correctly in the presence or after occurence of faults or stress conditions. From this point of view robustness engineering complements performance engineering to cover the full range of conditions to which a system may be exposed. Performance and robustness properties need to be continuously monitored during the development process to ensure that the system will meet the user's needs at the end. This paper will discuss aspects and problems of performance and robustness engineering. Also, it presents an approach, called "ISG" (Instantaneous System and Software Generation) which allows to continuously derive performance and robustness properties from the system-under-development. In case of ISG figures are derived from the real system right from the beginning. Therefore deviations from the desired functional, performance and robustness envelope can be corrected at an early stage. The capability for getting an immediate feedback from the system is obtained by the automated generation of the software from system engineering inputs. ISG builds the software by construction rules. This reduces the manual effort and allows for an immediate and representative feedback right after provision of inputs by the user. Due to automation the system can easily be instrumented on a user's demand without requiring any additional programming effort. ISG automatically stimulates the system and exposes it to stress tests and fault injection, and records coverage and performance figures. By incremental development a smooth transition from the first idea to the final version is possible at low risk. The ISG approach has been applied to the domain of real-time, distributed, fault-tolerant systems and shall be extended towards other application domains in future such as databases and graphical user interfaces.
收起
摘要 :
Statistical testing is of increasing interest because it allows full test automation – from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However,...
展开
Statistical testing is of increasing interest because it allows full test automation – from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However, automatically generated tests based on a statistical approach have to cope with the "oracle problem" and the "small target problem". The oracle problem represents the fact that software cannot conclude on the correctness of the derived results by itself, while the small target problem consists in the challenge of hitting sporadic, but important test conditions in a large input domain. In the worst case this problem can result in zeroprobability for specific test conditions. Although deterministic testing methods in principle do not suffer from the small target problem, they suffer from an oracle problem as well, which is the problem to know which test cases are needed. In general, the test criteria need to represent all viable questions a test engineer wants to pose on the respective system under test. Therefore deterministic methods are best at pointing out anticipated faults (as far as “anticipated” by a test engineer) while statistical methods can also reveal non-anticipated faults. BSSE has started already more than ten years ago with first activities in statistical testing, and built related tools. Based on the feedback from recent activities an approach has been defined aiming to overcome the weakness of statistical and deterministic testing by making a synthesis of the best of the two worlds. This approach includes automatic test case generation based on the information in the source code (prototype specification and code structure), optimisation by improved test criteria for statistical testing and methods known from deterministic testing,automated test evaluation and comprehensive presentation of results.
收起
摘要 :
Statistical testing is of increasing interest because it allows full test automation – from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However,...
展开
Statistical testing is of increasing interest because it allows full test automation – from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However, automatically generated tests based on a statistical approach have to cope with the "oracle problem" and the "small target problem". The oracle problem represents the fact that software cannot conclude on the correctness of the derived results by itself, while the small target problem consists in the challenge of hitting sporadic, but important test conditions in a large input domain. In the worst case this problem can result in zeroprobability for specific test conditions. Although deterministic testing methods in principle do not suffer from the small target problem, they suffer from an oracle problem as well, which is the problem to know which test cases are needed. In general, the test criteria need to represent all viable questions a test engineer wants to pose on the respective system under test. Therefore deterministic methods are best at pointing out anticipated faults (as far as “anticipated” by a test engineer) while statistical methods can also reveal non-anticipated faults. BSSE has started already more than ten years ago with first activities in statistical testing, and built related tools. Based on the feedback from recent activities an approach has been defined aiming to overcome the weakness of statistical and deterministic testing by making a synthesis of the best of the two worlds. This approach includes automatic test case generation based on the information in the source code (prototype specification and code structure), optimisation by improved test criteria for statistical testing and methods known from deterministic testing,automated test evaluation and comprehensive presentation of results.
收起
摘要 :
Statistical testing is of increasing interest because it allows full test automation - from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However, ...
展开
Statistical testing is of increasing interest because it allows full test automation - from test generation to evaluation - and hence reduces significantly the human test effort, while allowing a much broader test range. However, automatically generated tests based on a statistical approach have to cope with the "oracle problem" and the "small target problem". The oracle problem represents the fact that software cannot conclude on the correctness of the derived results by itself, while the small target problem consists in the challenge of hitting sporadic, but important test conditions in a large input domain. In the worst case this problem can result in zero-probability for specific test conditions. Although deterministic testing methods in principle do not suffer from the small target problem, they suffer from an oracle problem as well, which is the problem to know which test cases are needed. In general, the test criteria need to represent all viable questions a test engineer wants to pose on the respective system under test. Therefore deterministic methods are best at pointing out anticipated faults (as far as "anticipated" by a test engineer) while statistical methods can also reveal non-anticipated faults.
收起
原文获取